Hannink et al. Systematic Reviews 2013, 2:52 
http://www.systematicreviewsjournal.eom/content/2/1/52 



Si systematic 
| REVIEWS 



RESEARCH Open Access 



A systematic review of individual patient data 
meta-analyses on surgical interventions 

Gerjon Hannink 1 *, Hein G Gooszen 1 , Cornelis JHM van Laarhoven 2 and Maroeska M Rovers 1,3 



Abstract 

Background: Compared to subgroup analyses in a single study or in a traditional meta-analysis, an individual 
patient data meta-analysis (IPDMA) offers important potential advantages. We studied how many IPDMAs report on 
surgical interventions, how many of those surgical IPDMAs perform subgroup analyses, and whether these 
subgroup analyses have changed decision-making in clinical practice. 

Methods: Surgical IPDMAs were identified using a comprehensive literature search. The last search was conducted 
on 24 April 2012. For each IPDMA included, we obtained information using a standardized data extraction form, 
and the quality of reporting was assessed. We also checked whether results were implemented in clinical 
guidelines. 

Results: Of all 583 identified IPDMAs, 22 (4%) reported on a surgical intervention. Eighteen (82%) of these IPDMAs 
presented subgroup analyses. Subgroups were mainly based on patient and disease characteristics. The median 
number of reported subgroup analyses was 3.5 (IQR 1.25-6.5). Statistical methods for subgroup analyses were 
mentioned in 11 (61%) surgical IPDMAs. 

Eleven (61%) of the 18 IPDMAs performing subgroup analyses reported a significant overall effect estimate, whereas 
six (33%) reported a non-significant one. Of the IPDMAs that reported non-significant overall results, three IPDMAs 
(50%) reported significant results in one or more subgroup analyses. Results remained significant in one or more 
subgroups in eight of the IPDMAs (73%) that reported a significant overall result. 

Eight (44%) of the 18 significant subgroups appeared to be implemented in clinical guidelines. The quality of 
reporting among surgical IPDMAs varied from low to high quality. 

Conclusion: Many of the surgical IPDMAs performed subgroup analyses, but overall treatment effects were more 
often emphasized than subgroup effects. Although, most surgical IPDMAs included in the present study have only 
recently been published, about half of the significant subgroups were already implemented in treatment guidelines. 



Background 

Surgery has advanced spectacularly in the past 50 years, 
but many advances have not come from carefully 
planned research using valid study designs [1]. Research 
on surgical interventions is associated with several meth- 
odological and practical challenges of which few, if any, 
apply only to surgery. Surgical innovation is especially 
demanding because many of these challenges coincide 
[2], Perhaps this situation leads many surgeons to view 
randomized controlled trials (RCTs), although theoretic- 
ally advantageous, to be too difficult and impractical to 
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undertake, and even worse, irrelevant to their practice 
because of concerns about generalizability [2,3]. 

The results of RCTs are usually implemented in prac- 
tice by either treating or testing all patients in case of a 
'positive' study or treating or testing no-one in case of a 
'negative' study. Clinicians intuitively know that this ap- 
proach is oversimplified because in reality some patients 
benefit more than average whereas others do not benefit. 
This may explain why around 50% of the RCTs perform 
subgroup analyses [4,5]. However, misleading claims 
about subgroup effects based on a single study are com- 
mon [6]. 

Investigating subgroups is a highly relevant, but com- 
plex topic because of two interrelated concerns: failure 
to detect a relevant subgroup effect (false negative), and 
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a misleading claim about a subgroup effect which in 
reality does not exist (false positive). Both of these prob- 
lems can lead to suboptimal care for patients. Subgroup 
effects have been extensively and fiercely debated in the 
clinical, epidemiological, and statistical literature, espe- 
cially in the context of single trials or traditional meta- 
analyses based on published summary results [7-11]. 

Individual patient data meta-analyses (IPDMAs) differ 
from traditional meta-analyses in that an IPD meta- 
analysis uses the 'raw data of individual patients from 
included studies instead of the published summary re- 
sults of studies in a traditional meta-analysis [12]. Com- 
pared to subgroup analyses in a single study or in a 
traditional meta-analysis, an IPDMA offers important 
potential advantages, such as: (1) increased possibilities 
to perform more complex statistical analyses that better 
match the underlying data; (2) more power compared 
to single studies and traditional meta-analyses; (3) higher 
validity of subgroup analyses by avoiding ecological bias 
and by taking the distribution of other patient charac- 
teristics into account; (4) improved flexibility and stan- 
dardization of defining subgroups across studies; and (5) 
opportunities to examine the consistency of subgroup 
effects across studies [13-17]. 

In this paper we present a systematic overview of 
all IPDMAs on surgical interventions published. We 
studied the number and types of subgroup analyses 
performed, and whether these subgroups analyses 
influenced decision-making in clinical practice. 

Methods 

Search 

A comprehensive literature search in PubMed, Embase, 
Web of Science, and the Cochrane Library was 
conducted to identify all IPDMAs of RCTs. The last 
search was conducted on 24 April 2012. Keywords used 
to develop our search strategy were 'individual patient 
data' and 'meta-analysis' (see Additional file 1 for de- 
tailed search strategy). 

Selection 

In first instance, titles and abstracts were screened to 
identify eligible IPDMAs. Selection of potential eligible 
IPDMAs was restricted to IPD obtained from RCTs 
comparing surgical interventions. Patients had to be ran- 
domized over a surgical intervention in at least one 
treatment arm, and the surgical procedures had to be 
performed under general, spinal, epidural, or regional 
anesthesia. IPDMAs regarding drug-eluting medical de- 
vices and surgical trials in which a drug was the com- 
parison were excluded. 

Full text papers were retrieved when meta-analytic 
techniques for individual patient data of RCTs were 
used. IPDMAs using the same dataset or combination 



of datasets, studying/addressing different questions/ 
subgroups were included. If obvious duplicate pa- 
pers were available, the most elaborate paper was 
included. 

Data extraction and analysis 

Data from all included surgical IPDMAs were extracted 
with respect to specific characteristics, that is, publica- 
tion year, number of included trials and patients, do- 
main, type of intervention, comparison, and outcome 
measured. Regarding the subgroups, number, type, justi- 
fication, statistical methods, and results in relation to 
the overall effect estimate were studied. 

We classified five types of subgroups, patient charac- 
teristics (for example, age or gender), disease character- 
istics (for example, severity or co-morbidity), household 
characteristics (for example, socioeconomic status or 
smoking), intervention characteristics (for example, type 
of intervention or dose), and methodological characteris- 
tics (for example, quality of included trials or trial 
effect). Justification for subgroups analyses was catego- 
rized as based on literature, clinical experience, bio- 
logical mechanism, or no justification. 

We also assessed the quality of reporting of all selected 
IPDMAs. IPDMAs on RCTs should be reported 
according to the Preferred Reporting Items for System- 
atic Reviews and Meta- Analyses (PRISMA) [18]. Since 
this guideline is not specific to IPDMAs, it has been sug- 
gested that some additional information should be 
reported, for instance why the IPDMA approach was ini- 
tiated, whether there was a protocol for the IPDMA pro- 
ject, and whether a one-step or a two-step analysis was 
performed [12]. We judged the quality of reporting 
based on the 18 criteria suggested by Riley et al. [12]. 
Two independent reviewers (GH and MMR) selected eli- 
gible surgical IPDMAs and extracted data (duplicate in- 
dependently). Any disagreements were resolved by 
consensus. 

Finally, we reviewed available clinical guidelines for 
recommendations based on significant results of sub- 
group analyses from IPDMAs to determine the extent to 
which these results were implemented in clinical guide- 
lines. We conducted a PubMed search for fields 'pa- 
tients' (for example, carotid stenosis), 'intervention' (for 
example, carotid stenting) and 'comparison' (for ex- 
ample, endarterectomy), extracted from IPDMAs with 
significant subgroup analyses, and limited our search to 
'Practice Guideline'. We only included publications in 
English. We also searched the National Library of 
Guidelines (http://guidance.nice.org.uk/), and the Na- 
tional Guideline Clearinghouse (http://www.guidelines. 
gov/). 

All steps in this review were carried out according to a 
pre-defined protocol (Additional file 2). 
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Results 

Search 

In the search for IPDMAs, 3,597 potential eligible papers 
were identified. After studying the abstracts, 583 papers, 
published between 1991 and 2012, indeed reported an 
IPDMA. After detailed evaluation, 22 (4%) IPDMAs 
reported on a surgical intervention and met our inclu- 
sion criteria (Figure lj Additional file 3). 

Of the 22 surgical IPDMAs, 12 focused on cardiovas- 
cular interventions, three on inguinal hernia repair, three 
on gynecological interventions, two on orthopedic inter- 
ventions, one on a gastroenterological intervention, and 
one on ventilation tubes for otitis media (Table 1). The 
surgical IPDMA papers were published between 2005 
and 2012. Eighteen (82%) of the 22 surgical IPDMAs 
tried to identify subgroups of patients that benefit more 
or less from the surgical intervention. 



The remaining non-surgical IPDMAs predominantly 
focused on cancer, cardiovascular disease, and diabetes, 
and most assessed whether a treatment or intervention 
was effective, often in subgroups of patients. Before 2000 
only a few IPDMAs were published, whereas a consider- 
able rise in the number of published IPDMAs is seen be- 
tween 2005 and 2012 (Figure 2). This growth is most 
likely the result of an increased awareness the potential 
advantages of IPDMAs, and the initiation of collabora- 
tions to specifically perform such studies. 

Summary of IPDMAs using IPD (or part of IPD) from the 
same trials 

Of the 12 IPDMAs that focused on a cardiovascular 
intervention, four IPDMAs [26-29] used individual pa- 
tient data from the same 22 trials (6,763 patients) evalu- 
ating the clinical effects of primary percutaneous coronary 



3,597 records identified through database searching 

- PubMed 2,015 records 

- Embase 921 records 

- Web of Science 522 records 
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1,850 records excluded 

- Cohort, case report, or case-control instead of RCT 

- Methodological or tutorial review 

- Objective differed too much from IPDMA 

- Duplicate publications 

- Conference abstracts 8< protocols 

- Animal studies 



541 IPDMA excluded 

- Non-surgical 



42 full-text surgical IPDMA 
retrieved for detailed evaluation 



20 surgical IPDMA excluded 

- No IPDMA 

- Duplicate publications 



22 surgical IPDMA included 



18 surgical IPDMA 
with subgroup analysis 



4 surgical IPDMA 
without subgroup analysis 



Figure 1 Flowchart of study selection process for IPDMA of surgical interventions. 



Table 1 Characteristics of the 22 identified surgical IPDMAs 



Author and year 



RCTs 

(n) 



Patients 

in) 



Patients 



Intervention 



Comparison 



Outcome 



Subgroups (n) 



Overall 
effect 



Significant 
effect 



estimate estimates 
in 



subgroups 

(n/W) 



n Q 

S s 

2 .§ 



3 " 



Jorgenson ef a/., 2007 7 
[19] 



2,091 



Hlatky ef al, 2009 [20] 10 7,812 



Daniels ef a/., 2010 5 862 
[21] 

Burzotta ef al., 2009 1 1 2,68f 

[22] 



Carotid Stenting 3 
Trialists'Collaboration, 
2010 [23] 



3,433 



Middleton ef al, 2010 
[24] 



1/ 



2,814 



Women with 

cervical 

insufficiency 



Cervical cerclage 



Patients with Coronary artery 
multivessel bypass graft 

coronary disease 



Patients with 
chronic pelvic 

pain 



Laparoscopic 
uterosacral nerve 
ablation (LUNA) 



Patients with ST- Percutaneous 

elevation coronary 

myocardial intervention with 

infarction (STEMI) thrombectomy 



Patients with 
symptomatic 
carotid stenosis 



Carotid stenting 



Patients with 
heavy menstrual 
bleeding 



Hysterectomy, 
endometrial 
destruction (1 st < 
2nd generation), 
levonorgestrel 



Expectant 
management, no 
cerclage 



Percutaneous 

coronary 

intervention 



No LUNA 



Standard 
percutaneous 
coronary 
intervention 



Endarterectomy 



Endometrial 
destruction (1st 8 
2nd generation), 
levonorgestrel 
releasing intra- 



Primary 

Pregnancy loss or neonatal death 
before discharge from hospita 

Secondary 

Preterm delivery and maternal 
morbidity 

All-cause mortality 



Derived measure of worst pain 
level experienced 

Primary 

All-cause mortality 
Secondary 

Survival free from Ml, TLR, orTVR, 
major adverse coronary events 
(MACE), death+MI 

Primary 

Any stroke or death 
Secondary 

Disabling stroke or death, all- 
cause death, any stroke, 
myocardial infarction, severe local 
hematoma, severe wound 
infection 



Dissatisfaction rates 



Obstetric history, cervica 
(2) 



length NS 



0/2 



Age, sex, diabetes, smoking, 
hypertension, 

hypercholesterolemia, PVD, 
stability of symptoms, previous 
Ml, heart failure, LV function, no. 
of diseased vessel, proximal LAD, 
balloon vs.stent (14) 

Presence of visual pathology, site 
of pain, age, parity (4) 

Manual vs. non-manua 
thrombectomy devices, diabetes, 
primary vs. rescue PCI, treated vs. 
non-treated with llb/llla-inhibitors, 
ischemic time, infarct-related 
artery, pre-PCI TIMI flow (7) 



Age, sex, diabetes, hypertension, 
SBP, hypercholesterolemia, 
smoking, coronary heart disease, 
peripheral artery disease, most 
recent ipsilateral ischemic event, 
history of stroke, degree of 
ipsilateral ischemic stroke, 
contralateral severe carotid 
stenosis or occlusion, treatment 
within 14 days, patients recruited 
per center, center recruitment 
rate (16) 

Uterine cavity length, age, 
presence of fibroids/polyps, 
parity, baseline bleeding score (5) 



MS 



2/14 



NS 



1/4 



Ml 



1/16 



1/5 
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Table 1 Characteristics of the 22 identified surgical IPDMAs (Continued) 



Mercado et at, 2005 4 3,051 
[25] 



Boersma et at., 2006 22 6,767 

[26] 



Timmer et at., 2007 19 6,315 
[27] 

de Boer era/., 2010 22 6,767 

[28] 



de Boer et a/., 201 1 22 6,767 

[29] 

Fox etat, 2010 [30] 3 5,467 



Patients with 
multi-system 
coronary artery 
disease 



Patients with 
acute myocardial 
infarction 

Patients with 
acute myocardia 
infarction 

Patients with 
acute myocardial 
infarction 



Patients with 
acute myocardia 
infarction 

Patients with 
non-ST-elevation 
myocardial 
infarction 



releasing intra- 
uterine system 
(MIRENA) 

Percutaneous 
coronary 
intervention with 
multiple stenting 



PC 



PCI 



Primary PCI 



Primary PCI 



Routine invasive 
strategy 



uterine system 
(MIRENA) 



Coronary artery 
bypass graft 



Fibrinolysis 

Fibrinolysis 
Fibrinolysis 



Fibrinolysis 



Primary 

Composite of death, Ml, or stroke 
at 1 year FU 

Secondary 

Death, composite of death or Ml, 
repeat revascularization, 
composite of death, Ml, stroke, 
and repeat revascularization 

All-cause mortality 



Death, recurrent Ml, death or 
recurrent Ml, stroke 



Primary 

All-cause mortality 

Secondary reMI, stroke, composite 
of all-cause mortality or reMI, 
composite of all-cause mortality, 
reMI, or stroke 

All-cause mortality 



Selective invasive Primary 
strategy 



Composite of CV death or non- 
fatal Ml 

Secondary 

A -cause death, non-fatal Ml 



Age, gender, diabetes, smoking, 
number of diseased vessels (5) 



MS 



Age, sex, diabetes, prior Ml, Ml 
location, heart rate, SBP, 
fibrinolytic agent, front-loaded 
tPA, site volume (11) 

Diabetes (1) 



Age (1) 



High-risk patients (1) 



High-risk groups based on 
baseline characteristics (1) 



0/5 



1/11 



0/1 



0/1 



0/1 



Damman era/., 2012 3 5,467 
[31] 



Damman ef a/., 2012 3 
[32] 



5,467 



Patients with 
non-ST-elevation 
myocardial 
infarction 

Patients with 
non-ST-elevation 
myocardial 
infarction 



Routine invasive 
strategy 



Routine invasive 
strategy 



Selective invasive Primary 
strategy 



Composite of CV death or non- 
fatal Ml, CV death, Ml 



Selective invasive All-cause mortality 
strategy 



Age (1) 



Procedure-related Ml, 
spontaneous Ml (2) 



1/1 



1/2 



3iau ef at., 2009 [33] 6 



123 



Primary 



2/3 



Table 1 Characteristics of the 22 identified surgical IPDMAs (Continued) 



Rovers ef al., 2005 7 1 ,234 
[34] 



Salerno ef al., 2007 4 305 
[35] 



Staples ef al., 201 1 2 209 

[36] 



McCormack ef at, 25 4,165 
2003 [37] 



Scott ef al, 2002 [38] 1 1 3,347 



EL) Hernia Trialists 
Collaboration, 2002 
[39] 



35 



Patients with 
symptomatic 
unilateral anterior 
cruciate ligament 
injury 

Children with 
otitis media with 
effusion 



Cirrhotic patients 
with refractory 
ascites 



Patients with 
osteoporotic 
vertebral 
compression 
fractures 

Patients with 
clinical diagnosis 
of groin hernia 
for whom 
surgical 

management was 

judged 

appropriate 



Patients with 
clinical diagnosis 
of groin hernia 
for whom 
surgical 

management was 

judged 

appropriate 



Patients with 
clinical diagnosis 
of groin hernia 
for whom 
surgical 



Reconstruction with Reconstruction 



patellar tendon 
autograft 



Short-term 
ventilation tubes 



with hamstring 
tendon autograft 



Watchful waiting 



Transjugular 
intrahepatic 
portosystemic shunt 
(TIPS) 



Vertebroplasty 



Paracentesis 



Sham 



Laparoscopic repair Open repair 

(Transabdominal 
preperitoneal repair 
CTAPP) or totally 
extraperitoneal 
repair (TEP)) 



Mesh technique 



Non-mesh 
technique 



Laparoscopic repair, 
mesh methods 



Open repair, non- 
mesh methods 



Positive pivot-shift test Secondary Gender, age at surgery, trial effect 
Positive Lachman test 



Mean time spent with effusion, 
hearing, language development 



Primary 

Death from any cause before LT 

Secondary 

Liver-related death 

Scores for pain and function 



Duration of operation, 'opposite' 
method initiated, conversion, 
hematoma, seroma, wound/ 
superficial infection, mesh/deep 
infection, port site hernia, vascular 
injury, visceral injury, length of 
hospital stay, time to return to 
usual activities, persisting pain, 
persisting numbness, hernia 
recurrence, known death within 
30 days of surgery 

Duration of operation, 'opposite' NA 
method initiated, conversion, 
hematoma, seroma, wound/ 
superficial infection, serious 
complications, length of 
postoperative hospital stay, time 
to return to usual activities, 
persisting pain, persisting 
numbness, hernia recurrence, 
known death 

Hernia recurrence, persisting pain NA 



Hearing level at baseline, history NS 
of acute otitis media, upper 
respiratory infections, attending 
day care, socioeconomic status, 
siblings, season, history of 
breastfeeding, parental smoking 
(9) 

NA S 



Onset of pain, pain scores at NS 
baseline (2) 



NA 



2/9 



MA 



0/2 



''•I A 



NA 



IA 



Table 1 Characteristics of the 22 identified surgical IPDMAs (Continued) 



management was 

judged 

appropriate 

Gregson ef ai, 2012 8 2,186 Patients with Surgery Conservative Unfavorable outcome Location of hematoma, time from NA 4/5 

[40] spontaneous treatment event, age, Glascow Coma Score, 

supratentorial volume of hematoma (5) 

intracerebra 

hemorrhage 
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Non-surgical IPDMAs 
I I Surgical IPDMAs 




Year of publication 

Figure 2 Number of applied IPDMA published up to April 2012,* as identified by a systematic review of PubMed, Embase, Web of 
Science, and the Cochrane Library. Thirty-seven IPDMAs published in 2012 were identified up to 24 April 2012, when the review 
was conducted. 



intervention versus in-hospital fibrinolysis. In three IPDMAs 
[30-32], comparing routine invasive strategies with se- 
lective invasive strategies in 5,467 patients with non- 
ST segment elevation acute coronary syndromes, data 
from the same three trials (FRISC II, ICTUS, and 
RITA-3) were used. In addition, of the three IPDMAs 
that focused on inguinal hernia repair one IPDMA 
[39] presented a combination of the data used in the 
other two IPDMAs [37,38]. 

Number, justification, type, and methods of subgroups 
analyses in surgical IPDMAs 

In 18 (82%) of the full set of surgical IPDMAs assessed, 
subgroup analyses were performed to examine whether 
certain patients benefit more from a specific treatment 
than others. The median number of subgroups reported 
in these IPDMAs was 3.5 (range, 1-16, IQR 5.25 (1.25- 
6.5)). In 12 (67%) of the 18 surgical IPDMAs that stud- 
ied subgroups a justification for subgroup analyses was 
mentioned. Scientific literature was used for justification 
in these studies. 

The types of subgroups studied varied. Fifteen (83%) 
IPDMAs studied patient characteristics, five (28%) stud- 
ied household characteristics, 15 (83%) studied disea- 
se characteristics, and six (33%) studied intervention- 
related subgroups. Subgroups related to study or trial 
effects were studied in three (17%) IPDMAs. No IPDMAs 
studied subgroups related to the quality of the included 
trials, for example concealment of allocation, blinding, or 
completeness of follow-up. 

Twelve (55%) IPDMAs stratified their analysis per trial 
before pooling the results (two-step analysis). A one-step 
analysis was performed in four (18%) IPDMAs. Statistical 



methods for subgroup analyses were mentioned in 11 
(61%) of the 18 IPDMAs performing subgroup analyses. 
All IPDMAs that mentioned statistical methods for sub- 
group analysis used interaction tests. 

Only five (28%) surgical IPDMAs mentioned the power 
of the subgroup analyses. Three IPDMAs reported that 
their studies were underpowered to detect subgroup ef- 
fects, one IPDMA reported that their study was well 
powered to detect subgroups effects, however, did fail to 
show differences in subgroups, and one IPDMA men- 
tioned differences in power between different subgroups, 
but not whether these were over- or underpowered. 

Eleven (61%) of the 18 IPDMAs performing subgroup 
analyses reported a significant overall effect estimate, 
whereas six (33%) reported a non-significant one. One 
IPDMA (6%) did not report an overall effect estimate 
and only presented results of subgroup analyses [40] . 

Of the IPDMAs that reported non-significant overall 
results, three IPDMAs (50%) reported significant results 
in one or more subgroup analyses. Results remained sig- 
nificant in one or more subgroups in eight of the 
IPDMAs (73%) that reported a significant overall result. 

Thirty-six (40%) of the total number of 90 subgroups ana- 
lyses were performed on a non-significant or inconclusive 
overall effect estimate, whereas 49 (54%) were performed on 
a significant overall effect estimate. The remaining five (6%) 
subgroups originated from the one IPDMA that did not re- 
port an overall effect estimate and only presented results of 
subgroup analyses, four out of these five subgroups being sig- 
nificant [40]. Of the subgroup analyses performed on non- 
significant overall results, five (14%) became significant. Nine 
(18%) of those performed on IPDMAs with a significant 
overall result remained significant. 
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Eight (67%) of the 12 surgical IPDMAs with significant 
subgroups reported on what the implications of these 
significant results of their subgroup analyses were for 
clinical practice. Mainly, the importance of differentiat- 
ing when evaluating the efficacy and safety of new med- 
ical and interventional treatments, and translating these 
findings in treatment recommendations were empha- 
sized. Moreover, it was reported that the influence of 
certain subgroups had not been reported previously, that 
findings concurred with recent recommendations or 
guidelines, and that subgroups not per se needed to be 
an exclusion criterion for treatment. Eight (44%) of the 
18 significant subgroups were implemented in clinical 
guidelines. 

Quality of reporting 

The quality of reporting of the surgical IPDMAs varied 
(Figure 3). More than half of the IPDMA failed to report 
whether or not there was a protocol for the IPDMA pro- 
ject available. The reason why the IPD approach was ini- 
tiated and the numbers of patients within each of the 
original studies were generally well reported. For 17 
(77%) IPDMAs, the process used to identify relevant 
studies for the IPDMA were reported. Details on the 
statistical analysis were reported in 16 (73%) IPDMAs, 
however, details on the identification process and statis- 
tical analysis were not described in one IPDMA (4%), 
and were unclear in the remaining five (23%) IPDMAs. 

Discussion 

Our systematic review of all IPDMAs on surgical inter- 
ventions published so far provides an overview of the 



potential advantages of IPDMAs (see Table 2 for exam- 
ples). In 18 (82%) of the full set of 22 surgical IPDMAs 
assessed, subgroup analyses were performed to examine 
whether certain patients benefit more from a specific 
treatment than others. Eight (67%) of the 12 surgical 
IPDMAs with significant subgroups reported on what 
the implications of their findings were for clinical prac- 
tice. Forty-four percent (8 out of 18) of the significant 
subgroups were implemented in clinical guidelines. 

Although many IPDMAs performed subgroup ana- 
lyses, the overall treatment was usually the main focus 
of the paper. Only occasionally subgroup analyses were 
emphasized. In surgical IPDMAs, similar to IPDMAs in 
general [42], subgroups were often based on patient and 
disease characteristics. The median number of sub- 
groups has been reported to range from 2 to 4, the max- 
imum number of subgroups from 15 to 50 [6,43-45], 
which is comparable to our findings. Justification of sub- 
group analyses, the methods used to perform subgroup 
analyses, and power calculations for performing sub- 
group analyses are often not reported in IPDMAs 
[6,43-48]. However, 11 (65%) of the IPDMAs included in 
our study justified at least one of the subgroups on 
which they reported, scientific literature being the mode 
of justification used. This is in line with other studies 
that found that clinical experience or biochemical justifi- 
cation is rare [44,46,49]. Others showed that the propor- 
tion of studies that used interaction tests for at least one 
of their subgroups ranges from 10% to 56% [6,43-48], 
which is slightly lower compared to our findings. So far, 
few studies mentioned the importance of the power of 
subgroup analyses [6,44,50,51], and reported that many 



Protocol for IPDMA project 
Ethics approval necessary 
Why IPD approach initiated 
Process to identify relevant studies 
How authors approached 
How many authors approached and provided data 
Number of authors that not provided data & reasons 
All IPD provided or only proportion 
Difference between studies providing and not providing data 
Number of patients in original studies 
Details and handling of missing data 
Details on in/excluding patients originally ex/included 
One or two step analysis and details 
How many patients from each study included in each analysis 
Assumptions of statistical models validated 
IPD data results comparable with published results 
How IPD and non-1 PD analysed together 
Robustness of analysis following in/exclusion of non-IPD data 




I Unclear/ not stated 



20 
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Figure 3 Quality of IPDMA reporting surgical interventions. Numbers inside bars are numbers of studies. 
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Table 2 Two examples of differences in conclusions with regard to how patient-level characteristics modify treatment 
effect 



Example 

Effectiveness of coronary artery bypass grafting vs. percutaneous 
coronary interventions for multivessel disease. 



Effectiveness of routine vs. selective invasive strategy in patients 
with non-ST-segment elevation acute coronary syndrome. 



Description 

A two-step meta-analysis of individual patient data from 7,812 patients included 
in 10 randomized trials comparing coronary artery bypass grafting (CABG) and 
percutaneous coronary intervention (PCI) in patients with multivessel coronary 
artery disease, showed a similar overall treatment effect on long-term mortality 
after CABG and PCI [20]. However, in diabetic patients mortality was substantially 
lower in the CABG group than in the PCI group (HR 0.70, 95% CI 0.56-0.87). 
Mortality was similar between groups in patients without diabetes (HR 0.98, 95% 
CI 0.86-1.12; P=0.01 for interaction). Patient age modified the effect of treatment 
on mortality with hazard ratios of 1.25 (95% CI 0.94-1.66) in patients aged <55 
years, 0.90 (95% CI 0.75-1.09) in patients aged 55-64 years, and 0.82 (95% CI 0.70- 
0.97) in patients aged >65 years (P=0.002 for interaction). Treatment effect was 
not modified by other subgroups. CABG might be a better option for patients 
aged >65 years and patients with diabetes since mortality was found to be 
lower in these subgroups. These results have been implemented in clinical 
guidelines [41], 

An individual patient data meta-analysis of three randomized trials of routine 
versus selective invasive strategies in patients with non-ST-segment elevation 
acute coronary syndrome showed that a routine invasive strategy resulted in 
significantly less cardiovascular deaths (CV deaths) or non-fatal myocardia 
infarctions (Mis) compared to selective invasive strategies [30]. The authors used 
patient's baseline characteristics to develop a multivariable risk prediction model. 
A simplified integer risk score was derived from the risk prediction model to 
predict a patient's 5-year probability of CV death or Ml, and the patients were 
categorized into three risk groups (low, intermediate, and high risk). 

The treatment effect was similar between groups in patients with low-risk (HR 
0.80 (95% CI 0.63-1.02)) and intermediate-risk (HR 0.81 (95% CI 0.66-1.01)) scores. 
In patients with high-risk scores treatment favored routine over selective invasive 
strategies (HR 0.68 (95% CI 0.53-0.86)). There were 2.0% (95% CI -4.1-0.1%) and 
3.8% (95% CI -7.4- -0.1%) absolute risk reductions in CV death or Ml in the low- 
and intermediate-risk groups and an 1 1.1% (95% CI -18.4- -3.8%) absolute risk 
reduction in the highest-risk patients. The multivariable risk prediction model has 
not yet been implemented in clinical guidelines. 



reports put too much emphasis on subgroup analyses 
that commonly lacked statistical power. This is in agree- 
ment with the results of the present study. 

To the best of our knowledge we are the first to study 
surgical IPDMAs, and illustrate the merits of this 
method within surgery. However, some potential limita- 
tions should also be discussed. First, our literature 
search for surgical IPDMAs was limited to IPDMAs 
with IPD obtained from RCTs comparing surgical inter- 
ventions, excluding IPDMAs regarding drug-eluting 
medical devices and surgical trials in which a drug was 
the comparison, and records were limited to the English 
language. We, however, believe that our review provides 
a good representation of the method within the surgical 
field. Second, reporting bias could not be entirely ex- 
cluded, since reporting of subgroup effects in scientific 
publications might be influenced by reviewers' and edi- 
tors' opinions. Third, as most studies mentioned mul- 
tiple subgroups, a clustering effect might occur for 
reporting on justification and statistical methods. There- 
fore, the results were reported on study level instead 
of individual subgroup level. Fourth, in the 12 IPDMAs 
of cardiovascular interventions reporting subgroup analyses, 



several studies included a same set of trials and the pat- 
tern of exploring heterogeneity among these studies 
might be similar, that is, there might be a clustering ef- 
fect. As this might impact the subsequent analyses as 
well as the conclusion, we performed a sensitivity ana- 
lysis. The outcomes and conclusions were not substan- 
tially influenced by the inclusion of studies using the 
same set of trials. Fifth, the time from publication to im- 
plementation of a result into a guideline or clinical prac- 
tice can be highly variable, and sometimes takes even 
more than 10 years [52]. Most surgical IPDMAs in- 
cluded in the present study have only recently been 
published, and time to possible implementation has 
been rather short. Therefore, we might have under- 
estimated the implementation of results from IPDMAs 
into guidelines and/or clinical practice. 

Conclusions 

One of the challenges in medicine is to rationally imple- 
ment available therapies in clinical practice, in the ap- 
propriate patients at the appropriate time. Findings from 
IPDMAs might provide insight into opportunities to im- 
prove evidence-based treatment decisions for patients. 
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IPDMA is an extremely powerful tool if used correctly 
and provides the most definitive synthesis of the avail- 
able evidence, also for potential subgroups. Despite the 
recommendations available on reporting clinical trials 
and meta-analyses, such as PRISMA [18], these guide- 
lines have not been specifically developed for IPDMAs. 
The development of a generally accepted guideline for 
reporting on IPDMAs including subgroup analyses 
should therefore be encouraged. This seems the only op- 
tion to really improve the reporting, analyses, and claims 
and applicability of subgroup effects in clinical research. 
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